Apparatus

ABSTRACT

An apparatus comprising at least one processor and at least one memory including computer program code the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform determining a change in position and/or orientation of an apparatus, and processing at least two audio signals dependent on the change in position and/or orientation to generate at least one output signal wherein the processing of the two audio signals dependent on the change in position and/or orientation produces the output signal comprising a representation of acoustic energy from a first direction.

The present invention relates to apparatus for processing of audiosignals. The invention further relates to, but is not limited to,apparatus for processing audio and speech signals in audio devices.

In portable electronic apparatus, video and audio capture applicationswhere the user can record social events are popular. In suchenvironments, background sound sources or noise can easily interferewith the target sound source. For example at a party or live concert,there may be a significant amount of background noise which the user ofthe apparatus does not wish to capture or at least suppress in order to“focus” the audio capture on the target sound source.

Such apparatus may use a directional microphone or microphone array tocapture the acoustic waves from a set direction or with a definedprofile and output them as electronic signals representing the audiosignals which then may be processed and transmitted to other devices orstored for later playback.

For example apparatus with two or more microphones may be used withadaptive filtering (in the form of variable gain and delay factorsapplied to the audio signals) from each of the microphones in an attemptto beamform the microphone array reception pattern to focus on theactivity directly in front of the apparatus and thus avoid capturingnoise or unwanted audio sources peripheral to the device.

Similarly as well as background noise suppression/target sourceenhancement the user of the apparatus may wish to produce a desiredstable mix of audio channels from the captured audio signals based onthe original direction of the apparatus relative to the target audiosource.

However, there may be circumstances where the user is not able tomaintain the apparatus direction. For example the user of the apparatusmay wish to video the surroundings whilst maintaining capturing thesound source. For example during a wedding the user may wish to capturethe vows spoken by the couple at the wedding and avoid capturing thecongregation audio sources, and keeping the couple central in the audiostage, but at the same time move the camera focus to capture video ofthe interior of the church. Typically as the user moves the apparatusand the camera the microphone array is also moved and the audio capturefocus and the audio mix balance on the couple is lost.

Furthermore there may be situations where the user is not physicallyable to maintain supporting the apparatus in the direction originallypointed, for example to avoid a potentially dangerous situation or wherethe user is physically tired from holding the apparatus at a fixedposition. Typically in such circumstances the focus and the audio mixbalance moves with the direction of the apparatus and would remix thecaptured audio signal wherever the device is pointed, even if this newdirection is not the target sound source.

This invention proceeds from the consideration that the use of furtherinformation, for example sensor information, may assist the apparatus inthe control of audio capture and thus, for example, assist in thereduction of noise of the captured audio signals or correct audiomixing.

Embodiments of the present invention aim to address the above problem.

There is provided according to a first aspect of the invention a methodcomprising: determining a change in position and/or orientation of anapparatus; and processing at least two audio signals dependent on thechange in position and/or orientation to generate at least one outputsignal wherein the processing of the two audio signals dependent on thechange in position and/or orientation produces the output signalcomprising a representation of acoustic energy from a first direction.

The change in position and/or orientation is preferably at least one of:a relative change in position and/or orientation with respect to atarget audio source; and an absolute change in position and/ororientation.

The change in position and/or orientation may comprise a change inrotational position.

The method may further comprise: generating for each audio signal atleast one signal processing parameter dependent on a first position ofthe apparatus; and processing the at least two audio signals to producean initial output signal comprising a representation of acoustic energyfrom the first direction.

Determining the change in position and/or orientation of an apparatuspreferably comprises: determining whether the change in position and/ororientation of an apparatus is greater than at least one predefinedvalue; and generating at least one signal processing parameter dependenton the at least one predefined value.

The method may further comprise: converting at least four ambisonic typeA-format signals into at least four ambisonic type B-format signals; andwherein processing at least two audio signals dependent on the change inposition and/or orientation to generate at least one output signal maycomprise applying a rotation vector to at least one ambisonic typeB-format signal, and the rotation vector further comprises an offsetcomponent dependent on the change in position of the apparatus.

The at least one audio signal may comprise at least one of: at leastfour ambisonic type A-format signals; at least four ambisonic typeB-format signals; and at least one audio signal captured from at leastone microphone.

The first direction is preferably defined by an orientation and a gainprofile.

Determining a change in position and/or orientation of the apparatus maycomprise determining a change in position and/or orientation from afirst time period to a second time period using at least one of: adigital compass; an accelerometer; a gyroscope; a camera; and anacoustic characteristic determiner.

Determining a change in position and/or orientation of the apparatususing the camera may comprise: detecting an object of interest in afirst image in the first time period; and detecting a displacement ofthe object of interest in a later image in the second time period.

According to a second aspect of the invention there is provided anapparatus comprising at least one processor and at least one memoryincluding computer program code the at least one memory and the computerprogram code configured to, with the at least one processor, cause theapparatus at least to perform: determining a change in position and/ororientation of an apparatus; and processing at least two audio signalsdependent on the change in position and/or orientation to generate atleast one output signal wherein the processing of the two audio signalsdependent on the change in position and/or orientation produces theoutput signal comprising a representation of acoustic energy from afirst direction.

The change in position and/or orientation is preferably at least one of:a relative change in position and/or orientation with respect to atarget audio source; and an absolute change in position and/ororientation.

The change in position and/or orientation preferably comprises a changein rotational position.

The at least one memory and the computer program code is configured to,with the at least one processor, cause the apparatus to preferablyfurther perform: generating for each audio signal at least one signalprocessing parameter dependent on a first position and/or orientation ofthe apparatus; processing the at least two audio signals to produce aninitial output signal comprising a representation of acoustic energyfrom the first direction.

Determining the change in position and/or orientation of an apparatusmay cause the apparatus at least to perform: determining whether thechange in position and/or orientation of the apparatus is greater thanat least one predefined value; and generating at least one signalprocessing parameter dependent on the at least one predefined value.

The at least one memory and the computer program code is configured to,with the at least one processor, cause the apparatus to preferablyfurther perform: converting at least four ambisonic type A-formatsignals into at least four ambisonic type B-format signals; and whereinprocessing at least two audio signals dependent on the change inposition and/or orientation to generate at least one output signal maycause the apparatus at least to perform applying a rotation vector to atleast one ambisonic type B-format signal, and the rotation vectorfurther comprises an offset component dependent on the change inposition and/or orientation of the apparatus.

The at least one audio signal may comprise at least one of: at leastfour ambisonic type A-format signals; at least four ambisonic typeB-format signals; and at least one audio signal captured from at leastone microphone.

The first direction is preferably defined by an orientation and a gainprofile.

Determining a change in position and/or orientation of the apparatus maycause the apparatus to further perform determining the change inposition and/or orientations from a first time period to a second timeperiod using at least one of: a digital compass; an accelerometer; agyroscope; a camera; an acoustic tracker; and an acoustic characteristicdeterminer.

Determining the change in position and/or orientation of the apparatususing the camera may cause the apparatus to further perform: detectingan object of interest in a first image in the first time period; anddetecting a displacement of the object of interest in a later image inthe second time period.

According to a third aspect of the invention there is provided anapparatus comprising: a sensor configured to determine a change inposition and/or orientation of the apparatus; and a processor configuredto process at least two audio signals dependent on the change inposition and/or orientation to generate at least one output signalwherein the processing of the two audio signals dependent on the changein position and/or orientation produces the output signal comprising arepresentation of acoustic energy from a first direction.

The sensor is preferably configured to determine a change in positionand/or orientation as a change in rotational position of the apparatus.

The processor is preferably further configured to generate for eachaudio signal at least one signal processing parameter dependent on afirst position and/or orientation of the apparatus; and process the atleast two audio signals to produce an initial output signal comprising arepresentation of acoustic energy from the first direction.

The processor preferably comprises: an ambisonic converter configured toconvert at least four ambisonic type A-format signals into at least fourambisonic type B-format signals; and a vector rotatator configured toprocess the at least two audio signals dependent on the change inposition and/or orientation to apply a rotation vector to at least oneambisonic type B-format signal, and the rotation vector furthercomprises an offset component dependent on the change in position and/ororientation of the apparatus.

The apparatus may comprise a microphone array configured to capture theat least one audio signal as at least one of: at least four ambisonictype A-format signals; at least four ambisonic type B-format signals;and at least one audio signal captured from at least one microphone ofthe microphone array.

The first direction is preferably defined by an orientation and a gainprofile.

The apparatus may further comprise at least one of: a digital compass;an accelerometer; a gyroscope; a camera; an acoustic tracker; and anacoustic characteristic determiner.

The camera may further determine the change in position and/ororientation of the apparatus by being configured to: detect an object ofinterest in a first image in a first time period; and detect adisplacement of the object of interest in a later image in a second timeperiod.

The acoustic characteristic determiner may further determine the changein position and/or orientation of the apparatus by being configured to:detect an acoustic characteristic for an object of interest in a firsttime period; and detect a displacement of the acoustic characteristicfor the object of interest in a later image in a second time period.

According to a fourth aspect of the invention there is provided anapparatus comprising: sensing means for determining a change in positionand/or orientation of the apparatus; and processing means for processingat least two audio signals dependent on the change in position and/ororientation to generate at least one output signal wherein theprocessing of the two audio signals dependent on the change in positionand/or orientation produces the output signal comprising arepresentation of acoustic energy from a first direction.

According to a fifth aspect of the invention there is provided acomputer-readable medium encoded with instructions that, when executedby a computer perform: determining a change in position and/ororientation of an apparatus; and processing at least two audio signalsdependent on the change in position and/or orientation to generate atleast one output signal wherein the processing of the two audio signalsdependent on the change in position and/or orientation produces theoutput signal comprising a representation of acoustic energy from afirst direction.

An electronic device may comprise apparatus as described above.

A chipset may comprise apparatus as described above.

BRIEF DESCRIPTION OF DRAWINGS

For better understanding of the present invention, reference will now bemade by way of example to the accompanying drawings in which:

FIG. 1 shows schematically an apparatus employing embodiments of theapplication;

FIGS. 2 a and 2 b show schematically two microphone configurationarrangement suitable for use in apparatus such as shown in FIG. 1implementing some embodiments of the application;

FIG. 3 shows schematically the apparatus shown in FIG. 1 in furtherdetail according to some embodiments; and

FIG. 4 shows a flow diagram illustrating the operation of the apparatusaccording to some embodiments of the application.

The following describes apparatus and methods for the provision ofenhancing audio capture and recording flexibility apparatus with amicrophone array. In this regard reference is first made to FIG. 1 whichshows a schematic block diagram of an exemplary electronic device 10 orapparatus, which may incorporate enhanced audio signal captureperformance components and methods. Although the following examples havebeen described with respect to audio capture apparatus it would beappreciated that the embodiments described may be used as part of anaudio/video capture apparatus audio sub-system. The embodiments of theapplication attempt to use sensor information to enhance the audiosignal capture of apparatus by being able to control a channel selectionor beamforming operation in order to maintain a ‘focus’ or targetedaudio direction independent of the apparatus actual orientation orangle. In other words in such embodiments as described below it ispossible to maintain a audio targeted direction or beamforming or evenchannel extraction relative to an absolute direction and independent ofthe apparatus. In some embodiments this absolute direction may change ifthe target sound source moves.

The apparatus 10 may for example be a mobile terminal or user equipmentfor a wireless communication system. In other embodiments the apparatusmay be any audio recorder/player, for example a mp3 player, mediaplayer, digital or audio recorder, digital video recorder equipped withsuitable microphone array and sensors as described below.

The apparatus 10 in some embodiments comprises an audio processor 21.The audio processor 21 may be configured to execute various programcodes. The implemented program codes may comprise an audio captureenhancement code.

The implemented program codes may be stored for example in a memory forretrieval by the audio processor whenever needed. The memory couldfurther provide a section for storing data, for example data that hasbeen processed in accordance with the embodiments.

The audio capture enhancement code may in embodiments be implemented atleast partially in hardware or firmware.

The audio processor 21 may be linked to a user interface (UI).

The user interface 15 may enable a user to input commands to theelectronic device 10, for example via a keypad, buttons or switchesand/or to obtain information from the apparatus 10, for example via adisplay (not shown). It would be understood that the user interface 15may furthermore in some embodiments be any suitable combination of inputand display technology, for example a touch screen display suitable forboth receiving inputs from the user and displaying information to theuser.

The apparatus 10 may in some embodiments further comprise at least twomicrophones in a microphone array 11 for inputting or capturing acousticwaves and outputting audio or speech signals to be processed accordingto embodiments of the application. The audio or speech signals may,according to some embodiments, be stored in a data section of the memoryfor later processing.

A corresponding program code or hardware to control the capture of audiosignals using the at least two microphones may be activated to this endby the user via the user interface 15.

The apparatus 10 in such embodiments may further comprise ananalogue-to-digital converter (ADC) 14 configured to convert the inputanalogue audio signals from the microphone array 11 into digital audiosignals and provide the digital audio signals to the processor 21. Insome embodiments the microphone elements themselves comprise a suitableanalogue-to-digital converter and thus output digital audio signalsdirectly to the processor 21.

With respect to FIGS. 2 a and 2 b, two example microphone arrayconfigurations are shown in further detail. With respect to FIG. 2 a, atwo dimensional omniorientation microphone array configuration is shown.The microphone configuration is such that each microphone is directedwith a primary axis 120° away from the other two microphones primaryaxis. In this configuration, only three microphones are required tocover a complete 360° two dimensional plane and provide a possibleomniorientational coverage from non-omnidirectional profile microphones.Thus a first microphone 101 a has a coverage or profile given by thegain profile 103 a which is directed along a primary axis 120° from thesecond microphone 101 b gain profile 103 b primary axis and also 120°from the third microphone 101 c gain profile 103 c primary axis.Similarly the second microphone 101 b gain profile 103 b primary axis isalso 120° from the third microphone 101 c gain profile 103 a primaryaxis. In such microphone configuration an omniorientation microphone (a2-D omnidirectional microphone) may be simulated by adding together allthree microphone outputs and directional X-axis and Y-axis microphonesby weighted combining of the three microphone outputs.

With respect to FIG. 2 b, a three dimensional “omnidirectional”microphone array configuration is shown where each microphone isconsidered to be directed perpendicular to a face of a tetrahedron. Thuswith only four microphones, a full 360° solid angle coverage or profileis provided. In other words the four microphones may be combined toproduce a synthesised omnidirectional audio signal even if they are donot have an omnidirectional audio signal capture profile themselves. Inthe following examples, a four microphone array configuration asdescribed may be used to enhance audio recording, however it would beappreciated that any suitable microphone configuration may be used withsimilar effect.

The audio output of the microphones may be considered to be theequivalent to an ambisonic A-format encoding. In this encoding formatthe first microphone 111 a may be the left, front, up (LFU) microphone,the second microphone 111 b may be the left, back, down (LBD)microphone, the third microphone 111 c may be the right, front, down(RFD) microphone and the fourth microphone 111 d may be the right, back,up (RBU) microphone according to the convention of labelling used inambisonic A-format encoding.

The apparatus 10 may in some embodiments receive the audio signals froma microphone array 11 not implemented physically on the electronicdevice. For example the microphone array may be implemented of aseparate device, such as a microphone boom. The microphone boom may thentransmit the audio signals to the apparatus 10 via transceiver.

The received audio data may in some embodiments be stored, instead ofbeing processed immediately, in the data section of the memory, forinstance for later processing and presentation or forwarding to anotherelectronic device. In such embodiments the apparatus may store sensorinformation associated with the audio data and be processed according inan offline mode.

Furthermore the apparatus 10 may comprise sensors or a sensor bank 16.The sensor bank 16 receives information about the environment in whichthe apparatus 10 is operating and passes this information to theprocessor 21 in order to affect the processing of the audio signal andin particular to affect the processor 21 in audio capture/recordingapplications. The sensor bank 16 may comprise at least one of thefollowing set of sensors.

The sensor bank 16 may in some embodiments comprise a camera module. Thecamera module may in some embodiments comprise at least one camerahaving a lens for focusing an image on to a digital image capture meanssuch as a charged coupled device (CCD). In other embodiments the digitalimage capture means may be any suitable image capturing device such ascomplementary metal oxide semiconductor (CMOS) image sensor. The cameramodule further comprises in some embodiments a lamp or light source forilluminating an object before capturing an image of the object. In otherembodiments the camera may be configured to perform infra-red and nearinfra-red sensing for low ambient light sensing. In some embodiments thesensor bank 16 comprises a position/orientation sensor. Theposition/orientation sensor in some embodiments may be implemented by a3-D digital compass or solid state compass configured to determine theapparatus orientation with respect to the horizontal axis and azimuthwith respect to the vertical axis. In some embodiments theposition/orientation sensor may be a 2-D compass configured to determinethe apparatus orientation with respect to the horizontal axis only.

In some other embodiments the position/orientation sensor may be atleast one accelerometer or gyroscope configured to determine a change inacceleration in at least one axis.

It is to be understood again that the structure of the apparatus 10could be supplemented and varied in many ways.

It would be appreciated that the schematic structures described in FIG.3 and the method steps in FIG. 4 represent only a part of the operationof a complete audio capture/recording chain comprising some embodimentsas exemplary shown implemented in the apparatus shown in FIG. 1. Withrespect to FIG. 3, a schematic view of the processor is shown in furtherdetail with respect to some embodiments of the application.

With respect to FIG. 4, the operations of the apparatus shown in FIG. 3are described in further detail.

The processor may in some embodiments comprise a target selector/tracker305. The target selector/tracker 305 is configured to initialize theaudio capture process dependent on the sensor input. In some embodimentsthe target selector/tracker 305 receives an input from the userinterface to start the capture process or in other embodiments selectingan object to which the apparatus may produce a ‘sound lock’ to. In someembodiments the input from the user interface may be a ‘rec’ or recordfunction which starts the recording of both the audio and the video databy the apparatus.

On detecting a valid capture initialization input the targetselector/tracker 305 may transmit a signal to initialize the ambisonicconverter 301 and also the channel extractor/beamformer 303. In someembodiments the target selector/tracker 305 may transmit a signal to thechannel extractor containing information or data about the apparatus'initial orientation.

As described previously the microphone array in some embodiments isconfigured to capture audio signals from each of the microphones in thearray. For the examples described hereafter the 3D microphone arrayshown in FIG. 2 b supplies the captured audio signals. However it wouldbe appreciated that fewer or more microphones and configurations otherthan the 3D tetrahedral structure shown in FIG. 2 b may be employed.

The microphone audio signals in some embodiments are passed to ananalogue-to-digital converter 14 which are converted into a digitalformat also known as a raw microphone datA-format or an ambisonicA-format signal.

The audio processor 21 is configured to receive the digital audio at anAmbisonic A-format (or raw microphone format) to B-Format converter.

The Ambisonic A-format to B-format converter 301 is configured toreceive the digital microphone array data from each of the microphoneswithin the microphone array such as those shown configured in FIG. 2 band generate, dependent on the initialization signal from the targetselector/tracker 305, a synthesised audio signal set which represents anoverall sound pressure level, the W digital signal, and a series oforthogonal projected sound pressure levels, the X, Y and Z digitalsignals.

In the B-format encoding, also known as first-order Ambisonics, soundinformation is encoded into four channels: W, X, Y and Z. The W channelis the non-directional mono component of the signal, corresponding tothe output of an omnidirectional microphone. The X, Y and Z channels arethe directional components in three dimensions. They, correspond to theoutputs of three figure-of-eight microphones, facing forward, to theleft, and upward respectively.

The B-format signals are thus based on a spherical harmonicdecomposition of the sound field and correspond to the sound pressure(W), and the three components of the pressure gradient (X, Y, and Z) ata point in space. Together, these approximate the sound field on asphere around the microphone; formally the first-order truncation of themultipole expansion. This is called “first-order” because W (the monosignal) is the zero-order information, corresponding to a sphere(constant function on the sphere), while X, Y, and Z are the first-orderterms (the dipoles), corresponding to the response of figure-of-eightmicrophones—as functions, to particular functions that are positive onhalf the sphere, and negative of the other half. This first-ordertruncation is an approximation of the overall sound field.

Any playback of B-format ambisound signals may be derived by using alinear combination of these four channels, where each signal isdependent on the actual position of the speaker in relation to thecenter of an imaginary sphere the surface of which passes through allavailable speakers. In more advanced decoding schemes, spatialequalization is in some embodiments applied to the signals to accountfor the differences in the high- and low-frequency sound localizationmechanisms in human hearing. A further refinement may account for thedistance of the listener from the loudspeakers.

The A-format to B-format converter 301 may therefore in some embodimentsgenerate a W, X, Y and Z format digital audio signal from the microphonecaptured LFU, FBD, RBU and RFD digital audio signals using the followingequations:

W=−(LFU+LBD+RBU+RFD)

X=2.83(−LFU+LBD+RBU−RFD)

Y=2.83(−LFU−LBD+RBU+RFD)

Z=2.83(−LFU+LBD−RBU+RFD)

The converted B-format digital audio signals may then be passed to thechannel extractor 303.

The channel extractor/beamformer 303, having received the ambisonicB-format audio signals, and the initialization signal from the targetselector/tracker then performs a beamforming or channel extraction toproduce the required number of audio output channels.

The channel extractor 303 in some embodiments determines the position ofeach of the required number of audio channels from a predetermined listof speaker orientations which contain data on where the required audiooutput channels are with respect to the apparatus.

The channel extractor 303 may then output audio signals reflecting therequired orientation using the following equation (assuming that thereis no Z channel component as the speakers are in the X-Y plane).

Pn=W+X cos(θn)+Y sin(θn),

where θn is the direction/orientation of the speaker (or channel)relative to the original apparatus orientation.

In other words the channel extractor 303 generates each of the channelsby calculating the angle between the front back (X axis or initialorientation of the apparatus) and the required channel orientation.

In some embodiments this may be calculated using a single calculationfor each channel.

In such embodiments, fixed point processing should be sufficient togenerate the channel audio signals and thus may be calculated quicklywithout need for floating point calculations. For example in someembodiments a look-up table may be used which would require only 720bits in total to store the correct cosine and sine values for aorientation step size of 1 degree.

In some embodiments the channel extractor may furthermore have thecosine and sine values for predefined audio format configurations. Forexample the following table of values may be used in a six channel audiosystem:

O/P W X Y L 0.5018 0.6218 0.4406 R 0.5018 0.6218 −0.4406 SL 0.8392−0.3692 0.5757 SR 0.8392 −0.3692 −0.5757 SL′ 0.4465 −0.1964 0.3063 SR′0.4465 −0.1964 −0.3063where L is left front channel, R is right front channel, SL is surroundleft (left rear) channel, SR is surround right (right rear), SL′ issecond surround left (left rear-mid) and SR′ is second surround right(right rear-mid).

The initial processing of the audio signal is shown in FIG. 4 by step403.

Furthermore the target selector/tracker 305 is configured to maintain atrack on the target position/orientation by monitoring the output of thesensor bank 16. In some embodiments the target selector/tracker 305determines from this information whether or not the apparatus has moved(in relation to the audio stage).

For example where the sensor bank 16 comprises a compass, for example a3-D or 2-D compass, the target selector/tracker 305 may receive thesensor data as a digital representation of the X-Y plane orientation.Any difference in sensor input may be converted to a suitable angleformat and used to determine whether the apparatus has moved.

Furthermore in some embodiments where the sensor bank 16 comprises anaccelerometer or gyroscope, the output from the accelerometer orgyroscope may be monitored by the target selector/tracker 305. Thetarget selector/tracker 305 may using relevant look up tables orprocessing detect any change in orientation or movement of theapparatus.

Further in some embodiments where the sensor bank 16 comprises a camerathe output of the camera may be monitored by the target selector/tracker305. In such embodiments a series of images captured at a first instancemay be processed to determine any points of interest in the images whichare located at a far distance and close to the original axis of theapparatus to the apparatus. For example the camera may determine a pointof interest when video/audio recording an orchestra event such as afixed light pattern from the stage or light reflections from a stablestructure such as a pillar, door, or similar. The targetselector/tracker 305 may then monitor further captured images todetermine movement of the point of interest from image to image todetermine an approximate angle of displacement of the audio capture orrecording beam direction.

In some embodiments the selection of the target source may be associatedwith a specific acoustic characteristic. For example in some embodimentsthe target selector/tracker may perform an acoustic fingerprinting ofthe source. An acoustic fingerprinting may for example identify aspecific relationship between a fundamental and harmonic frequencies ofthe target source. The target tracker/selector may then in someembodiments track the target source by monitoring any movement in theacoustic properties of the signals from the microphones.

Such an acoustic characteristic determiner may thus in some embodimentsdetermine the change in position and/or orientation of the apparatus bybeing configured to: detect an acoustic characteristic for an object ofinterest in a first time period; and detect a displacement of theacoustic characteristic for the object of interest in a later image in asecond time period. In other embodiments any suitable acoustic trackingoperation or components may be employed to assist in the tracking of theobject of interest.

In some embodiments the determination of movement is a threshold event.In other words only when a sufficiently large movement is detected thenthe target selector/tracker is triggered to output any furtherinformation to the channel extractor/beamformer 303. In some otherembodiments the determination of movement is continuous and any changeis detected and affects a change in the position/angle output passed tothe channel extractor/beamformer 303.

The determination of movement is shown in FIG. 4 by step 405.

If no or insufficient movement is detected then the target trackercontinues to monitor the sensors and the channel extractor/beamformer303 continues to process the B-format audio signals using the samespeaker orientations.

If no or insufficient movement is detected by the targetselector/tracker 305 then the target selector/tracker 305 passes anorientation offset value or orientation absolute value to the channelextractor/beamformer 303. The channel extractor/beamformer may thenprocess the B-format audio signals using the speaker orientations withthe orientation offset value or new absolute speaker orientation value.

For example in such embodiments any detected change in angle of theapparatus δθ may then be passed to the channel extractor and the valuesof the channels recalculated using a new value of θ, θ_(new)=θ+δθ.

Thus in such embodiments, the apparatus may produce an accurate andcontinuous representation of the sound stage even when the apparatusmoves.

Thus in some embodiments an audio “image” stabilisation is achievedwhere the audio stage is stabilised independent of motion of theapparatus.

The audio processor 21 furthermore may output the audio channel data insome embodiments for further processing to process the audio signalaccording to any suitable audio processing algorithm to produce a moreefficiently encoded data stream suitable for storage or transmission.For example in some embodiments the audio processor 21 may furtherprocess the Ambisonic format signal to convert it into a further format.

Although the above examples have been described with respect to 2Dtarget selection and tracking It would be appreciated that a similarapproach may be used for 3D target selection and tracking, for examplethe target tracker 503 outputs orientation angles θ representing the X-Yorientation, and φ representing the Z orientation. In other words the 2Dor 3D compass or accelerometer may be used together or separately toproduce compensation data for audio targeting compensation.

Furthermore although the above examples use the ambisonic audio formatany suitable digital audio format may be used with a suitablebeamforming processing. For example the beamforming/channel extractionoperation may apply a finite impulse response (FIR) or infinite impulseresponse (IIR) digital filter to each microphone input signal.

The finite impulse digital filters may be pure gain (in other words withno memory) or gain and delay filtering of the digital microphone audiosignals.

Although the above examples describe embodiments of the inventionoperating within an electronic device 10 or apparatus, it would beappreciated that the invention as described below may be implemented aspart of any audio processor.

Thus, for example, embodiments of the invention may be implemented in anaudio processor which may implement audio processing over fixed or wiredcommunication paths.

Thus user equipment may comprise an audio processor such as thosedescribed in embodiments of the invention above.

It shall be appreciated that the term user equipment is intended tocover any suitable type of wireless user equipment, such as mobiletelephones, portable data processing devices or portable web browsers.

In general, the various embodiments of the invention may be implementedin hardware or special purpose circuits, software, logic or anycombination thereof.

For example, some aspects may be implemented in hardware, while otheraspects may be implemented in firmware or software which may be executedby a controller, microprocessor or other computing device, although theinvention is not limited thereto. While various aspects of the inventionmay be illustrated and described as block diagrams, flow charts, orusing some other pictorial representation, it is well understood thatthese blocks, apparatus, systems, techniques or methods described hereinmay be implemented in, as non-limiting examples, hardware, software,firmware, special purpose circuits or logic, general purpose hardware orcontroller or other computing devices, or some combination thereof.

Therefore in summary there is in at least one embodiment an apparatuscomprising: a sensor configured to determine a change in position and/ororientation of the apparatus; and a processor configured to process atleast two audio signals dependent on the change in position and/ororientation to generate at least one output signal wherein the processormay be configured to process the two audio signals dependent on thechange in position and/or orientation to produce the output signalcomprising a representation of acoustic energy from a first direction.

Or in some embodiments there may be an apparatus comprising at least oneprocessor and at least one memory including computer program code the atleast one memory and the computer program code configured to, with theat least one processor, cause the apparatus at least to perform:determining a change in position and/or orientation of an apparatus; andprocessing at least two audio signals dependent on the change inposition and/or orientation to generate at least one output signalwherein the processing of the two audio signals dependent on the changein position and/or orientation produces the output signal comprising arepresentation of acoustic energy from a first direction.

The embodiments of this invention may be implemented by computersoftware executable by a data processor of the mobile device, such as inthe processor entity, or by hardware, or by a combination of softwareand hardware. Further in this regard it should be noted that any blocksof the logic flow as in the Figures may represent program steps, orinterconnected logic circuits, blocks and functions, or a combination ofprogram steps and logic circuits, blocks and functions. The software maybe stored on such physical media as memory chips, or memory blocksimplemented within the processor, magnetic media such as hard disk orfloppy disks, and optical media such as for example DVD and the datavariants thereof, CD.

Thus at least one embodiment comprises a computer-readable mediumencoded with instructions that, when executed by a computer perform:determining a change in position and/or orientation of the apparatus;and processing at least two audio signals dependent on the change inposition and/or orientation to generate at least one output signalwherein the processing of the two audio signals dependent on the changein position and/or orientation produces the output signal comprising arepresentation of acoustic energy from a first direction.

The memory may be of any type suitable to the local technicalenvironment and may be implemented using any suitable data storagetechnology, such as semiconductor-based memory devices, magnetic memorydevices and systems, optical memory devices and systems, fixed memoryand removable memory. The data processors may be of any type suitable tothe local technical environment, and may include one or more of generalpurpose computers, special purpose computers, microprocessors, digitalsignal processors (DSPs), application specific integrated circuits(ASIC), gate level circuits and processors based on multi-core processorarchitecture, as non-limiting examples.

Embodiments of the inventions may be practiced in various componentssuch as integrated circuit modules. The design of integrated circuits isby and large a highly automated process. Complex and powerful softwaretools are available for converting a logic level design into asemiconductor circuit design ready to be etched and formed on asemiconductor substrate.

Programs, such as those provided by Synopsys, Inc. of Mountain View,Calif. and Cadence Design, of San Jose, Calif. automatically routeconductors and locate components on a semiconductor chip using wellestablished rules of design as well as libraries of pre-stored designmodules. Once the design for a semiconductor circuit has been completed,the resultant design, in a standardized electronic format (e.g., Opus,GDSII, or the like) may be transmitted to a semiconductor fabricationfacility or “fab” for fabrication.

As used in this application, the term ‘circuitry’ refers to all of thefollowing:

-   -   (a) hardware-only circuit implementations (such as        implementations in only analog and/or digital circuitry) and    -   (b) to combinations of circuits and software (and/or firmware),        such as: (i) to a combination of processor(s) or (ii) to        portions of processor(s)/software (including digital signal        processor(s)), software, and memory(ies) that work together to        cause an apparatus, such as a mobile phone or server, to perform        various functions and    -   (c) to circuits, such as a microprocessor(s) or a portion of a        microprocessor(s), that require software or firmware for        operation, even if the software or firmware is not physically        present.

This definition of ‘circuitry’ applies to all uses of this term in thisapplication, including any claims. As a further example, as used in thisapplication, the term ‘circuitry’ would also cover an implementation ofmerely a processor (or multiple processors) or portion of a processorand its (or their) accompanying software and/or firmware. The term‘circuitry’ would also cover, for example and if applicable to theparticular claim element, a baseband integrated circuit or applicationsprocessor integrated circuit for a mobile phone or similar integratedcircuit in server, a cellular network device, or other network device.

The foregoing description has provided by way of exemplary andnon-limiting examples a full and informative description of theexemplary embodiment of this invention. However, various modificationsand adaptations may become apparent to those skilled in the relevantarts in view of the foregoing description, when read in conjunction withthe accompanying drawings and the appended claims. However, all such andsimilar modifications of the teachings of this invention will still fallwithin the scope of this invention as defined in the appended claims.

1. A method comprising: determining a change in position and/ororientation of an apparatus; and processing at least two audio signalsdependent on the change in position and/or orientation to generate atleast one output signal wherein the processing of the two audio signalsdependent on the change in position and/or orientation produces theoutput signal comprising a representation of acoustic energy from afirst direction.
 2. The method as claimed in claim 1, wherein the changein position and/or orientation is at least one of: a relative change inposition and/or orientation with respect to a target audio source; andan absolute change in position and/or orientation of the apparatus. 3.The method as claimed in claim 1, wherein the change in position and/ororientation comprises a change in rotational position of the apparatus.4. The method as claimed in claim 1, further comprising: generating foreach audio signal at least one signal processing parameter dependent ona first position of the apparatus; processing the at least two audiosignals to produce an initial output signal comprising a representationof acoustic energy from the first direction.
 5. The method as claimed inclaim 1, wherein determining the change in position and/or orientationof an apparatus comprises: determining whether the change in positionand/or orientation of an apparatus is greater than at least onepredefined value; and generating at least one signal processingparameter dependent on the at least one predefined value.
 6. The methodas claimed in claim 1, further comprising: converting a first formatsignal into a second format signal; and wherein processing at least twoaudio signals dependent on the change in position and/or orientation togenerate at least one output signal comprises applying a rotation vectorto the second format signal, and the rotation vector further comprisesan offset component dependent on the change in position of theapparatus.
 7. The method as claimed in claim 1, wherein at least one ofthe at least two audio signals comprises at least one of: at least fourambisonic type A-format signals; at least four ambisonic type B-formatsignals; and at least one audio signal captured from at least onemicrophone.
 8. The method as claimed in claim 1, wherein the firstdirection is defined by an orientation and a gain profile.
 9. The methodas claimed in claim 1, wherein determining a change in position and/ororientation of the apparatus comprises determining a change in positionand/or orientation from a first time period to a second time periodusing at least one of: a digital compass; an accelerometer; a gyroscopean acoustic tracker; and a camera.
 10. The method as claimed in claim 9,wherein determining a change in position and/or orientation of theapparatus using the camera comprises: detecting an object of interest ina first image in the first time period; and detecting a displacement ofthe object of interest in a later image in the second time period. 11.An apparatus comprising at least one processor and at least one memoryincluding computer program code the at least one memory and the computerprogram code configured to, with the at least one processor, causes theapparatus at least to: determine a change in position and/or orientationof an apparatus; and process at least two audio signals dependent on thechange in position and/or orientation to generate at least one outputsignal wherein causing the apparatus to process the two audio signalsdependent on the change in position and/or orientation produces theoutput signal comprising a representation of acoustic energy from afirst direction.
 12. The apparatus as claimed in claim 11, wherein thechange in position and/or orientation is at least one of: a relativechange in position and/or orientation with respect to an target audiosource; and an absolute change in position and/or orientation of theapparatus.
 13. The apparatus as claimed in claim 11, wherein the changein position and/or orientation comprises a change in rotational positionof the apparatus.
 14. The apparatus as claimed in claim 11, wherein theat least one memory and the computer program code is configured to, withthe at least one processor, causes the apparatus to: generate for eachaudio signal at least one signal processing parameter dependent on afirst position and/or orientation of the apparatus; process the at leasttwo audio signals to produce an initial output signal comprising arepresentation of acoustic energy from the first direction.
 15. Theapparatus as claimed in claim 11, wherein causing the apparatus todetermine the change in position and/or orientation causes the apparatusat least to: determine whether the change in position and/or orientationof the apparatus is greater than at least one predefined value; andgenerate at least one signal processing parameter dependent on the atleast one predefined value.
 16. The apparatus as claimed in claim 11,wherein the at least one memory and the computer program code isconfigured to, with the at least one processor, causes the apparatus to:convert a first format signal into a second format signal; and whereincausing the apparatus to process at least two audio signals dependent onthe change in position and/or orientation to generate at least oneoutput signal causes the apparatus at least to apply a rotation vectorto the second format signal, and the rotation vector further comprisesan offset component dependent on the change in position and/ororientation of the apparatus.
 17. The apparatus as claimed in claim 11,wherein at least one of the at least two audio signals comprises atleast one of: at least four ambisonic type A-format signals; at leastfour ambisonic type B-format signals; and at least one audio signalcaptured from at least one microphone.
 18. The apparatus as claimed inclaim 11, wherein the first direction is defined by an orientation and again profile.
 19. The apparatus as claimed in claim 11, wherein causingthe apparatus to determine a change in position and/or orientation ofthe apparatus causes the apparatus to further determine the change inposition and/or orientations from a first time period to a second timeperiod using at least one of: a digital compass; an accelerometer; agyroscope an acoustic tracker; and a camera.
 20. The apparatus asclaimed in claim 19, wherein causing the apparatus to determine thechange in position and/or orientation of the apparatus using the cameracauses the apparatus to further: detect an object of interest in a firstimage in the first time period; and detect a displacement of the objectof interest in a later image in the second time period.